import tensorflow as tf
Small error above -- in calculating the partial derivative of $f$ with respect to $x$, I wrote $\delta a / \delta f$ right before the equals sign when it should have been $\delta a / \delta x$.
x = tf.Variable(-2.0)
y = tf.Variable(1.0)
z = tf.Variable(3.0)
with tf.GradientTape() as tape:
tape.watch(x)
f = x * y + 2 * z + (1 / x)
df_dx = tape.gradient(f,x)
print("df/dx: ", df_dx)
with tf.GradientTape() as tape:
tape.watch(y)
f = x * y + 2 * z + (1 / x)
df_dy = tape.gradient(f,y)
print("df/dy: ", df_dy)
with tf.GradientTape() as tape:
tape.watch(z)
f = x * y + 2 * z + (1 / x)
df_dz = tape.gradient(f,z)
print("df/dz: ", df_dz)
We can see from the above calculations that the Tensorflow outputs agree with our hand-calculated values.
x = tf.Variable(1.0)
y = tf.Variable(-3.0)
with tf.GradientTape() as tape:
tape.watch(x)
f = (x + tf.sigmoid(-y)) / ((x - y)**2)
df_dx = tape.gradient(f,x)
print("df/dx: ", df_dx)
with tf.GradientTape() as tape:
tape.watch(y)
f = (x + tf.sigmoid(-y)) / ((x - y)**2)
df_dy = tape.gradient(f,y)
print("df/dy: ", df_dy)
Our partial derivaties match up here as well!
The only small caveat is that they're both off by roughly $.0002$, which is reasonable given that we did some rounding with our hand-calculatons.